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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

• If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

!)□ Responsive to Gonnmunication(s) filed on . 



2a)n This action is FINAL. 2b)[3 This action is non-final. 

3) 0 Since this application is in condition for allowance except for fonnal matters, prosecution as to the merits is 

closed in accordance with the practice under Expa/te Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) K Claim(s) 1-21 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) 6-11. 21 is/are allowed. 

6) 13 Claim(s) 1-5,12, 18-20 is/are rejected. 

7) 13 Claim(s) 13-17 is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: a)n accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1, 85(a). 

11) 0 The proposed drawing correction filed on is: a)n approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) n The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)nAII b)n Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 

Attachment(5) 

1 ) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). 



2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-1 52) 

3) 13 Information Disclosure Statement(s) (PTO-1 449) Paper No(s) 6^. 6) d Other: 
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Art Unit: 2674 

DETAILED ACTION 
Information Disclosure Statement 

1. The information disclosure statement filed 10/1 1/00 as paper # 3 is not found in the file 
record . The examiner proposes duplicate sent by Applicant via Examiner faxsimile (703 746 
5850). 

Claim Objections 

2. Claim 21 is objected to because of the following informalities: improper phrase 
{entire?partial?} in page 29, claim 21, line 5. Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the invention 
thereof by the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act of 1999 
(AIPA) do not apply to the examination of this application as the application being examined 
was not (1) filed on or after November 29, 2000, or (2) voluntarily published under 35 U.S.C. 
122(b). Therefore, this application is examined under 35 U.S.C. 102(e) prior to the amendment 
by the AIPA (pre-AIPA 35 U.S.C. 102(e)). 



i 
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4. Claims 1, 18-19 are rejected under 35 U.S.C. 102(e) as being anticipated by Norton et 
al. (US Patent 5,704,836) . 

As to claims 1 and 18, Norton et al. discloses an input device ( command system 20, column 8, 
lines 7-18 ) for providing a signal to effect translational and/or rotational movement of an 
object on a graphical display , associated method and associated article of manufacture, 
comprising: 

a device for capturing video images ( optical detector unit 22 as shown in figure 1, 
column 9 , lines 2-8); 

an input image processor ( subject positioning unit 46 as shown in figure 2) 

programmed to 

(a) isolate the human form from the background in the captured video image ( 
area of the frame 90 is isolated from buffer zone 96, column 12, lines 1-8) 

(b) determine the position and movement of the human body ( movement of 
arm, leg, hand, column 2, lines 37-41) 

(c) generate an output signal responsive to the position and/or movement of the 
human arms (movement of user left arm causing a character in a game to move to 
left indicates a translational movement, column 25, lines 15-17); and 

an output image processor programmed to effect translational and or/rotational movement 
of an object on a graphical display in response to the signals received from the input 
image processor (control system 34 as shown in figure 2, column 10, lines 59-66). 
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As to claim 19, the article of manufacture of claim 18 wherein the signal generated by the 
program of instructions is used to generate the state of a flight simulator graphical display ( see 
the same citation for claim 18 ). It is noted that intended use of a flight simulator is still within 
scope of a graphical display taught by Norton et al ( display unit 36 including a graphical 
user interface incorporating any type of graphic, column 8, lines 25-34) 
Claim Rejections - 35 USC §103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 2-3 are rejected under 35 U.S.C. 103(a) as being unpatentable over Norton et al. 
in view of Freeman ( US Patent 5,454,043) . 

As to claim 2, Norton et al. teaches the input device of claim 1 wherein the output image 
processor changes the graphical display but does not teach said change according to the 
perspective of what a flying object would see. Freeman teaches gesture recognition by a video 
camera that scans the hand 16 of an individual 18 , determining the position of the icon ( 
column 5, lines 61-67, figure 1 showing a flying object 10 moving in direction similar to 
that of hand gesture in motion of an individual). It would have been obvious to a person of 
ordinary skill in the art at the time of the invention to utilize the apparatus taught by 
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Norton and modify the recognition software to include the gesture recognition which 
changes the display of an icon that denotes a flying object as taught by Freeman to obtain 
the apparatus Norton et al. modified by Freeman, because it will result in realistic 
simulation of an object on display that is in flying motion. 

As to claim 3, Norton et al. teaches the input device of claim 1 ( see same citation for claim 1 ) 
wherein the output image processor generates a graphical display but fails to teach graphical 
display of a flying object whose position and motion are responsive to the signal output by the 
input image processor. Freeman teaches gesture recognition by a video camera that scans the 
hand 16 of an individual 18 that determines the position of the icon ( column 5, lines 61-67, 
figure 1 showing a flying object 10 moving in direction similar to that of hand gesture in 
motion of an individual). It would have been obvious to a person of ordinary skill in the art 
at the time of the invention to utilize the apparatus taught by Norton and modify the 
recognition software to include the gesture recognition which changes the display of an 
icon that denotes a flying object as taught by Freeman to obtain the apparatus Norton et 
al. modified by Freeman, because it will result in realistic simulation of an object on display 
that is in flying motion. Note that apparatus Norton modified by Freeman now teaches a 
flying object ( Freeman, airplane icon 10 of figure 1) with position and motion dependent 
to signal output ( hand gesture of individual) by the aforementioned input image 
processor. 
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7. Claims 4-5, 12 and 20 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Norton et al. in view of Tsutsuguchi et al. ( US Patent 6,072,494) . 

As to claim 4, Norton et al. teaches a method for generating signals to effect translational 
and/or rotational movements of an object on a graphical display ( display unit 36 including a 
graphical user interface incorporating any type of graphic, column 8, lines 25-34) , 

comprising: 

providing an image processor and a device for capturing video images ( subject 
positioning unit 46 as shown in figure 2, optical detector unit 22 as shown in figure 1, 
column 9 , lines 2-8); 

capturing video images with the device and processing those images to isolate a human 
form from a background (area of the frame 90 is isolated from buffer zone 96, column 12, 
lines 1-8) 

However, Norton et al. fails to teach steps for calculating the arm position and movement data: 
generating a signal responsive to the arm position and movement data for effecting translational 
and/or rotational movement of an object on a graphical display. Tsutsuguchi et al. teaches steps 
for calculating arm position and movement data ( joint position determination 22 and angle 
calculation 24 of figures 2 and S, column 3, lines 9-65, column 5, lines 47-67, column 8, lines 
8-24 )• It would have been obvious to a person of ordinary skill in the art at the time of the 
invention to utilize the method taught by Norton et al., then modify image processing 
software to include routine software for calculating arm position and movement data as 
taught by Tsutsuguchi et al. to obtain the combined method Norton et al. modified by 
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Tsutsuguchi et al., because it would enhance precision of translational and/or rotational 
movement of an object on display. Note that method Norton et al. modified by Tsutsuguchi 
et al. now generates a signal responsive to the arm position and movement data for 
effecting translational and/or rotational movement of an object on a graphical display. 

Note also that method Norton et al. modified by Tsutsuguchi et al. teaches step for 
isolate the arm portions of the human form ( Norton et al., area of the frame 90 is isolated 
from buffer zone 96, column 12, lines 1-8, note Tsutsuguchi et al. teaches motion modeling 
selected according with processing efficiency and/or required reality, column 8, lines 40-42 

); 

As to claims 5 and 20, Norton et al. teaches a method for generating signals to effect 
translational and/or rotational movements of an object on a graphical display ( display unit 36 
including a graphical user interface incorporating any type of graphic, column 8, lines 25- 

34) , comprising: 

providing an image processor and a device for capturing video images ( subject 
positioning unit 46 as shown in figure 2, optical detector unit 22 as shown in figure 1, 
column 9 , lines 2-8); 

capturing video images with the device and processing those images to isolate a human 
form from a background (area of the frame 90 is isolated from buffer zone 96, column 12, 
lines 1-8) 

Hov^ever, Norton et al. fails to teach steps for calculating the arm position and movement data: 
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generating a signal responsive to the arm position and movement data for effecting translational 
and/or rotational movement of an object on a graphical display. Tsutsuguchi et al. teaches steps 
for calculating arm position and movement data ( joint position determination 22 and angle 
calculation 24 of figures 2 and 5, column 3, lines 9-65, column 5, lines 47-67, column 8, lines 
8-24 ). It would have been obvious to a person of ordinary skill in the art at the time of the 
invention to utilize the method taught by Norton et ah, then modify image processing 
software to include routine software for calculating arm position and movement data as 
taught by Tsutsuguchi et al. to obtain the combined method Norton et al. modified by 
Tsutsuguchi et aL, because it would enhance precision of translational and/or rotational 
movement of an object on display. Note that method Norton et al. modified by Tsutsuguchi 
et al. now generates a signal responsive to the arm position and movement data using the 
image processor for effecting translational and/or rotational movement of an object on a 
graphical display. 

Note also that method Norton et al. modified by Tsutsuguchi et al. teaches step for 
isolate the arm portions of the human form from a captured video image using the image 
processor (Norton et al, area of the frame 90 is isolated from buffer zone 96, column 12, 
lines 1-8, Tsutsuguchi et al. teaches animation generation apparatus 20 in figure 2) 

As to claim 12, claim 12 differs from claim 5 by phrases "for use in a flight simulator graphical 
display", "for use in generating the state of a flight simulator graphical display" ( see the same 
citation for claim 5 because it is noted that intended use of a flight simulator is still within scope 
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of a graphical display taught by Norton et al ( display unit 36 including a graphical user 
interface incorporating any type of graphic, column 8, lines 25-34) . 



Allowable Subject Matter 
8. Claims 6-1 1, 21 are allowed. 

The following is a statement of reasons for the indication of allowable subject matter: 
As to claims 6-1 1 and 21, none of prior art discloses a method for generating signals to effect 
translational and/or rotational movements of an object using human arm position and/or 
movement data, comprising isolating a view comprising a foreground object subject image view 
by performing an algorithm on the video sequence and the frame that does not include the 
person, computing the arm angles by calculating angles of principle moment of the nonzero 
pixels in the arm portions of the video image. Tsutsuguchi et al. only teaches arm angle 
calculation using linear interpolation and nonlinear interpolation techniques. 
Claims 13-17 are objected to as being dependent upon a rejected base claim, but would be 
allowable if rewritten in independent form including all of the limitations of the base claim and 
any intervening claims. 

As to claims 13, 15-17, none of prior art teaches a method for generating signals to effect 
translational and/or rotational movements of an object using human arm position and/or 
movement data, wherein the flight simulator graphical display includes as an object a flying 
creature that moves wings. 

As to claim 14, none of prior art teaches a method for generating signals to effect translational 
and/or rotational movements of an object using human arm position and/or movement data. 
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wherein the flight simulator graphical display depicts a change in perspective of what a flying 
creature would see. 

Conclusion 

9. The prior art of made of record is not relied upon, but pertinent to Applicant's 
disclosure: 

US Patent Petrich et al. 6, 104,379 

Reference Petrich et al. is made of record as it discloses a forearm supported 
exoskeleton hand-tracking device 

The nonprior art of made of record is not relied upon, but pertinent to Applicant's 
disclosure: 

US Patent AppHcation Publication Russell et al. US2002/0135581 Al 

Reference Russell et al. is made of record as it discloses a method and system for 
controlling an avatar using computer vision wherein arm movement of a user acts as effector 
information to control an avatar on display. 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to FRANCIS N NGUYEN whose telephone number is 703 308- 
8858. The examiner can normally be reached during hours 8:00 AM- 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, RICHARD A HJERPE can be reached at 703 305-4579. 
Any response to this action should be mailed to: 
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Commissioner of Patents and Trademarks 
Washington, D.C. 20231 
or faxed to: 

(703) 872-9314 ( for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington, VA, Sixth Floor ( Receptionist). 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Technology Center 2600 Customer Service whose telephone number is 
(703) 306-0377. 




FRANCIS N NGUYEN 

Examiner 

Art Unit 2674 



FN 

November 22nd, 2002 



